Repository navigation
Bound shutdown waits and retain Python cleanup - #25
ChrisKrawczyk wants to merge 8 commits into
Conversation
Preserve the prepared core and startup failures across repeated lifecycle calls, validate shutdown deadlines before mutation, and distinguish timeout from operational failure while retaining successful None return values. Add supervised lifecycle regressions, test-hook wiring, and migration docs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 73748d4f-2eb6-47d2-9dbc-af8681e80e83
Preserve upstream replay isolation, failed-race semantics, trailing lock options, release metadata, and the Rust toolchain pin. Integrate options into protected startup preparation and add raw/typed lifecycle race regressions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 73748d4f-2eb6-47d2-9dbc-af8681e80e83
|
Conflict reconciliation is pushed in b8c3213 (a normal merge of main; no history rewrite). GitHub now reports this draft conflict-free. Upstream replay/race fixes, trailing lock-option parameters, release metadata and the toolchain pin are retained, with option preparation still inside the protected lifecycle startup path. The requested independent Claude Opus 5 integration review covered all four lifecycle PRs and returned PASS, with no blocking findings. The reviewed tree is exactly the committed tree; microsoft/duroxide#64 is unchanged. Non-blocking follow-ups include choosing a new sibling package version before release and the existing signed-to-unsigned negative-option validation gap. This supersedes the original conflict note only: the published core/provider graph, final registry pins, PostgreSQL, non-Windows/five-RID, broader regressions and hosted CI remain unwaived gates. The PR stays draft; nothing was merged, released or published. |
Retain one native cleanup operation independently of bounded callers and preserve the first SDK budget, terminal state and Python error shapes. Integrate upstream release/replay changes and exercise cancellation-safe provider futures separately from non-yielding native work. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 73748d4f-2eb6-47d2-9dbc-af8681e80e83
|
Upstream-first reconciliation is pushed in beec0e7 (normal merge of main; no history rewrite). GitHub reports this draft conflict-free. The adapter retains one native cleanup operation and separately bounds repeated observations against the first SDK budget, preserving successful None returns and Python error shapes. Upstream 0.2.1 release metadata and replay/KV fixes are retained. Tests distinguish cancellable fetch futures from non-yielding native frames; no guarantee for unregistered Python work was added. The independent scoped four-repository review is PASS, including the corrected inherited .NET concurrency-test defect. Python's instrumented lifecycle suite passed 20 cases, including 100 repetitions per force ordering and isolated timeout/late-retirement probes. Production lifecycle/lock/replay/typed selections passed 26 with nine expected hook/POSIX skips. Production assets are restored; the reviewed source tree exactly matches the commit. Core microsoft/duroxide#64 now uses upstream task tracking and is about 32% smaller by feature changed-line count. The local core 0.1.32/provider 0.1.36 graph is development-only: published core lacks additive APIs. Registry delivery, PostgreSQL, non-Windows/five-RID, broader regression and hosted CI remain OPEN. The PR stays draft; nothing was merged, released or published. |
…rage Separate retained shutdown results from callback-bearing ownership. Prove collection after quiescence and strengthen startup, registration, timing and subprocess diagnostic regressions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 73748d4f-2eb6-47d2-9dbc-af8681e80e83
Keep lifecycle changes under Unreleased while adopting main version metadata and duroxide-pg 0.1.37. Verify both native modes on Python 3.12 and 3.13 against the retained local-core graph; compatible core publication remains a separate gate. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 73748d4f-2eb6-47d2-9dbc-af8681e80e83
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 73748d4f-2eb6-47d2-9dbc-af8681e80e83
Expose string-valued stop state and independently owned synchronous observers. Release the GIL during waits, preserve finite timeout deadlines, propagate signals and support cross-thread close without worker shutdown. Cover real faults, collection and notification lifetime, and document blocking/asyncio boundaries. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 73748d4f-2eb6-47d2-9dbc-af8681e80e83
Explain the qualified core cleanup policy and synchronous callback ownership separately from detached work and passive stop observation. Preserve existing shutdown budgets and interruption semantics. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 73748d4f-2eb6-47d2-9dbc-af8681e80e83
Summary
Bound Python shutdown waits without discarding native cleanup, and expose passive stop observation. Successful shutdown still returns
None.RuntimeErrorvalues without changing registration policy; distinguish shutdownTimeoutError.runtime.stop_observer()metadata observation and isolated lifecycle/observation controls, repeated races and native provenance.The latest commit,
f427a32, updates three documents after real-binding qualification against the cleanup-policy implementation in microsoft/duroxide#64 atcc06a9e. No production Python correction, dependency change or new public API was needed for that increment. The earlier lifecycle/observation implementation and upstream reconciliation remain in this PR.flowchart TD Stop["First valid stop"] --> Cleanup["Keep native cleanup owned"] Stop --> Wait["Wait within grace plus 5000 ms"] Cleanup -->|Cleanup proven| Release["Release obsolete core and activity-callback ownership"] Cleanup -->|Cleanup unproven| Retain["Retain ownership"] Wait -->|Total expires first| Timeout["Raise TimeoutError; keep cleanup owned"] Timeout --> Terminate["Application or supervisor must terminate the process"]Behavior and review focus
Omitted grace remains 1000 ms, with a total of grace plus 5000 ms. Zero requests force, not instant cleanup. Pre-start shutdown is terminal. Serialize lifecycle/registration calls on each Runtime, including calls from other threads while the GIL is released. Metrics remain unavailable after stop or accepted startup failure.
Timeout still requires application- or supervisor-owned process termination, even after later cleanup. The SDK never exits the process itself.
RuntimeErrororlifecycle_shutdown_failedalone does not prove cleanup; owner loss still requires retained ownership. PyO3 defers off-GIL reference-count decrements until the next GIL entry. Independent providers and Clients keep their lifetimes, and the global orchestration-handler map is unchanged.A synchronous Python callback stays owned until it returns, even if shutdown times out. Timeout/native drop cannot preempt it. Application-created threads and other unregistered work remain separately owned; this binding does not gain .NET's foreign-completion barrier.
The matching core contains only named user-cleanup failures. Eligible late destruction preserves actual outer outcomes or valid finalized decisions, not an inner success followed by outer failure. Core, reporting, finalization and registered-cleanup failures remain fatal. Valid output/decisions do not establish provider commit.
Contained incidents use native tracing target
duroxide::runtime::cleanup, categorycontained_user_cleanup; they do not themselves setFaulted. Stop observation is not an incident feed or a health check. At most two protected payload-disposal attempts can still block;cleanup-skippedmeans an allocation was left undisposed. Containment does not repair state or recover process abort. Returned lifecycle errors exclude panic payloads, but the separate process-wide Rust panic hook can print them.stop_observer()retains metadata rather than Runtime/provider/registry ownership. Readstatebefore synchronouswait_for_change(last_seen, timeout_ms=None). The wait releases the GIL and checks Python signals; timeout andKeyboardInterruptdo not request shutdown. Each observer permits one pending wait. Closing it from another thread wakes the wait and reports observer closure. Source closure delivers an unseen final state beforeNone; neither closure nor a status proves cleanup. Async callers must offload blocking waits and arrange finite waits/closure rather than assuming asyncio cancellation cancels native observation.The lifecycle guide
documents driver preconditions, observation and ownership. The latest documentation increment preserves executable examples and diagrams.
Current cleanup-policy qualification: November 18-20
Two fresh Windows x64 development/unwind native builds used the actual maturin path and separate Python 3.12 environments with the reviewed core implementation.
Compiled features and loaded extension hashes were checked; production test-only exports are absent. Probes cover contained versus fatal failures, actual generator disposal/valid decisions, inner-success/outer-failure behavior, synchronous callbacks and separately created application threads.
The held synchronous callback stayed owned through a measured 5.016-second zero-grace shutdown timeout; the original
5 <= elapsedSeconds < 6assertion was unchanged. Existing missing-token cancellation behavior remains false. No budget or watchdog was relaxed.The source-built graph uses modified core declaring 0.1.32, Tokio 1.54.1, futures-util 0.3.34 and resolved PG 0.1.38; the manifest declares PG 0.1.37. Original pins, locks and native assets are unchanged. Published core 0.1.32 is not equivalent to that modified source.
The full cross-SDK selection passed 109 distinct cases / 112 executions across six native builds, with no final failures, skips, zero selections, aborts or watchdog timeouts. No production SDK correction was needed. This is local SQLite/in-memory development evidence, not Python 3.13, optimized, PostgreSQL, platform or installed-package qualification of the new policy.
The current core selection passed 372 distinct cases / 712 executions. The documentation audit and independent review/challenge found no blocking defect; optional Runtime-lifetime correlation for cleanup incidents was not added.
Earlier evidence and remaining gates
Before the new policy, reconciliation checks on Python 3.12 and 3.13 each passed 30 hook-free lifecycle cases with 9 expected hook-only skips and 39 instrumented cases. Python 3.12 additionally passed 42 PostgreSQL-backed E2E cases including the SQLite control; instrumented all-target Clippy passed. Both ordered races retained 100 iterations. That earlier graph used registry PG 0.1.37. Those dev builds and PostgreSQL results are historical, not reruns of this policy. The temporary database container and credentials were removed afterward.
Later passive-observation evidence is summarized in microsoft/duroxide#64, separately from the fresh cleanup-policy matrix.
Depends on microsoft/duroxide#64. The earlier hosted registry failure records the missing published lifecycle API dependency. Current-head CI must be assessed separately.
Compatible core publication under maintainer authorization, final registry minima/provider graph/locks/native assets, and hosted, optimized/release-profile, supported-platform, PostgreSQL and installed-package checks remain gates. A provider version alone does not qualify a shared production database migration.
No release, package publication, publishing dispatch or merge into main was performed or requested. This delivery does not change dependency overrides or locks, and commits no native extensions, workflow artifacts or unrelated work.
Generated with PAW.